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ABSTRACT 

— >By  employing  a  recently  obtained  error  bound  for  differentiable  convex 
inequalities,  it  is  shown  that,  under  appropriate  constraint  qualifications, 
a  minimum  solution  of  an  exact  penalty  function  for  a  single  value  of  the 
penalty  parameter  which  exceeds  a  certain  threshold,  is  also  a  solution  of 
the  convex  program  associated  with  the  penalty  function.  No  a  priori  as¬ 
sumption  is  made  regarding  the  solvability  of  the  convex  program.  If  such 
a  solvability  assumption  is  made  then  we  show  that  a  threshold  value  of  the 
penalty  parameter  can  be  used  which  is  smaller  than  both  the  above-mentioned 
value  and  that  of  Zangwill.  These  various  threshold  values  of  the  penalty 
parameter  also  apply  to  the  well  known  big-M  method  of  linear  programming.  <c _ _ 
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SIGNIFICANCE  AND  EXPLANATION 


By  lumping  constraints  In  a  certain  way  Into  the  objective  function 
to  be  minimized,  constrained  optimization  problems  can  be  solved  as 
simple  unconstrained  optimization  problems.  The  principal  contribution 
of  this  work  Is  to  show  the  validity  of  this  approach  without  a  priori 
assuming  that  the  constrained  optimization  problem  Is  solvable  and 
without  requiring  the  penalty  parameter  to  take  on  more  than  one  value. 


The  responsibility  for  the  wording  and  views  expressed  In  this  descriptive 
summary  lies  with  MRC,  and  not  with  the  author  of  this  report. 


SUFFICIENCY  OF  EXACT  PENALTY  MINIMIZATION 


0.  L.  Mangasarlan 


1 .  Introduction 

Consider  the  convex  program 


(1.1)  minimize  f(x)  subject  to  g(x)  <  0 


where  f:  Rn  ■*  R,  g:  Rn  Rn  are  convex  functions  on  the  n-dlmenslonal 
real  Euclidean  space  Rn.  It  Is  well  known  [11,6]  that  If  (1.1)  has  a 
solution  x  and  If  the  constraints  of  (1.1)  satisfy  a  constraint  qual¬ 
ification  then  the  exact  penalty  function 

m 

(1.2)  P(x,o):-  f (x)  +  aeg(x)+  -  f(x)  +  a  j  max  (0,  g^x)} 


where  e  Is  a  vector  of  ones  In  R?,  has  a  global  minimum  at  x  for  each 
value  of  a  >  a  for  some  threshold  value  a.  In  [11,  p.  356;  2,  Theorem  40] 
It  was  shown  that 


(1.3) 


fix1)  -  f(x)  +  1 

min  -  g^x1) 
Ul^m 


where  x1  Is  any  point  satisfying  the  Slater  constraint  qualification 


(1.4) 


gU1)  <  o 


In  [6,  Theorem  4.9]  It  was  shown  that 

(1.5)  a  ■  a-:»  ||u||„  ■  max  u 

!<1<m  1 
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where  u  is  an  optimal  Lagrange  multiplier  for  (1.1)  provided  that  (1.4) 
holds.  A  minor  modification  of  the  proof  of  [6,  Theorem  4.9]  which  Invokes 
[10,  Theorem  28.2]  Instead  of  [7,  Theorem  5.4.8]  extends  (1.5)  to  the  case 
where  a  relaxed  Slater  constraint  qualification  holds,  that  is 


0.6) 


9T  (x2)  <  0,  gT  (x2)  <  0  for 
l\  l2 


some  x 


where  g^  Is  nonlinear  and  gj  is  linear  and  ^  u  I?  ■  {l,...,m}.  In 

contrast  Zangwlll's  threshold  (1.3)  does  not  hold  under  the  relaxed  Slater 
constraint  qualification  (1.6)  but  must  be  replaced  by  a  different  value 
given  by  (2.2)  below. 

What  Is  not  well  known  and  constitutes  a  principal  concern  of  this  work 
are  converses  to  the  results  stated  above.  In  [11,  p.  356;  2,  Theorem  40] 
Zangwlll  shows  that  If  we  assume  a  priori  that  the  minimization  problem  (1.1) 
has  a  solution,  the  Slater  constraint  qualification  (1.4)  Is  satisfied  and 
x  minimizes  the  exact  penalty  function  (1.2)  for  some  a^a-j,  then  x 
solves  the  minimization  problem  (1.1).  Note  the  a  priori  assumptions  that 
(1.1)  Is  solvable  and  that  It  satisfies  the  Slater  constraint  qualification. 
By  contrast  in  [6,  Theorem  4.1]  without  any  a  priori  assumptions  regarding 
the  solvability  of  the  minimization  problem  (1.1)  or  the  satisfaction  of  a 
constraint  qualification  it  was  shown  that  if  (1.1)  is  feasible,  that  is 
g(x)  <  0  for  some  x,  and  if  x  minimizes  P(x,a)  for  all  values  of 
a  >a  for  some  a,  then  x  solves  the  minimization  problem  (1.1).  Note 
the  distinction  between  these  two  sufficient  conditions  for  x  to  solve 
the  minimization  problem  (1.1).  In  Zangwlll's  result  there  are  a  priori 
assumptions  that  (1.1)  is  solvable  and  that  its  constraints  satisfy  the 
Slater  constraint  qualification,  while  the  penalty  function  P(x,a)  need 


.  >  V  V 


-  -  .v  s.  , 


fTT 


be  minimized  for  a  single  value  of  a  >  i|.  In  [6,  Theorem  4.1]  no  a  priori 

assumption  regarding  the  existence  of  a  solution  to  (1.1)  Is  made,  however 

feasibility  of  (1.1)  Is  assumed  and  x  must  be  a  solution  to  min  P(x,a) 

x*Rn 

for  al1_  a  >  a  for  some  a.  In  order  for  x  to  be  a  solution  to  (1.1). 

A  primary  purpose  of  this  work  Is  to  combine  the  good  features  of  these 

two  results,  namely  the  minimization  of  the  penalty  function  for  a  single 

value  of  the  penalty  parameter  and  without  an  a  priori  assumption  that  the 

minimization  problem  has  a  solution.  This  Is  done  In  Theorem  3.1  where  It 

Is  established  that  If  for  a  single  value  of  the  penalty  parameter  o  >  oij 

for  a  well  defined  a^,  x  minimizes  the  exact  penalty  function  P(x,a) 

over  Rn,  then  x  Is  also  a  global  solution  of  the  minimization  problem 

(1.1).  Although  no  a  priori  assumption  regarding  the  solvability  of  (1.1) 

Is  made  In  Theorem  3.1,  both  the  relaxed  Slater  constraint  qualification 

(1.6)  and  a  mild  asymptotic  constraint  qualification  (3.2)  are  needed  In 

order  to  Invoke  the  recent  [8,  Theorem  2.1]  absolute  error  bound  for  convex 

differentiable  Inequalities  which  plays  a  key  role  In  the  derivation  of 

* 

Theorem  3.1.  Another  result  of  this  work  Is  a  two-way  Improvement  of 
Zangwlll's  sufficiency  result  in  Theorem  2.1,  where  the  threshold  value  of 

a  Is  decreased  from  cij  of  (1.3)  to  of  (1.5)  and  the  Slater  con¬ 

straint  qualification  (1.4)  Is  replaced  by  the  relaxed  constraint  qualifi¬ 
cation  (1.6).  We  also  give  In  Corollary  2.3  a  finite  counterpart  of  the 
threshold  value  Oj  of  (1.3)  when  the  Slater  constraint  qualification  (1.4) 
Is  replaced  by  the  relaxed  qualification  (1.6)  which  renders  5^  Infinite. 
Table  1  below  gives  a  general  outline  of  the  relations  between  the  various 
sufficiency  results  derived  here  and  elsewhere  for  exact  penalty  functions 
and  Indicates  the  key  assumptions  needed  for  the  different  results  to  hold. 


A  priori  solvability 

Assumed 

of  min.  prob.  (1 .1 ) : 

Not  assumed 

1 1  » 

N  1  A| 

S  1  0 

-  1  - 

(M 

-  1 

Han-Mangasarlan 
[6,  Theorem  4.1] 

Constraint  qu 

Not  assumed 

c  '0 

1  2. 

5  1  Al 

1  2! 

tJ  1  0 

e 

Zangwlll  [11 ,  p.  356] 

3r  1  2 
«/>  “ 

1  £ 

*  1  f 

Theorem  2.1 

Theorem  3.1 

i  |  § 

c  1  < 

1 

£  1 

. 

1 

Table  1:  An  outline  of  the  key  assumptions  needed  In  the 
various  sufficiency  theorems  establishing  that 
each  mlnlmlzer  of  an  exact  penalty  function  (1.2) 
solves  the  minimization  problem  (1.1). 


In  Section  3  of  the  paper  we  show  that  the  blg-M  method  of  linear 
programming  [1,9]  Is  in  fact  equivalent  to  an  exact  penalty  problem  and 
hence  the  threshold  values  of  the  penalty  parameter  developed  In  this  work 
apply  to  It  as  well  as  to  a  blg-M  formulation  for  convex  programs.  Such 
threshold  values  do  not  seem  to  have  been  given  for  the  blg-M  method  for 
linear  programs. 

We  briefly  describe  now  our  notation.  For  a  vector  x  in  the 
n-dlmenslonal  real  Euclidean  space  Rn,  x+  will  denote  the  vector  in  Rn 


.  V 


v 

aiU  iili  bi  f 


with  components  (x+)^  »  max  {x^,0},  1-1,. ...n.  For  a  vector  norm  j|x|| 
on  Rn,  ]| x ||  will  denote  the  dual  norm  on  Rn,  that  Is  ||x||' »  max  xy. 


where  xy  denotes  the  scalar  product  t  x.y4.  The  Cauchy-Schwarz 

1-1  1  1 


■  . 

Inequality  |xy|  <  ||x||  •  ||y||  for  x  and  y  In  R  follows  Immediately 


1  ,  1 


from  this  definition  of  the  dual  norm.  For  1  <  p,  q  <  «  and  —  +  j-  ■  1, 


1 


the  p-norm  lix Hp:*  ^  I  |xj|*>)*)  anc*  the  q-norm  are  dual  norms  In  Rn.  For 
an  mxn  matrix  A,  denotes  the  1th  row,  while  || A]| p  denotes  the  matrix 


norm  subordinate  to  the  vector  norm  J|  •  ||  .  that  Is  || A ||  -  max  ||Ax||  . 


p'  . p  iixiTp-i — p 

The  consistency  condition  j|Ax||p  <  ||A||p  ||x||p  follows  Immediately  from 
this  definition  of  a  matrix  norm.  We  shall  also  use  ||*||  to  denote  an 
arbitrary  vector  norm  and  Its  subordinate  matrix  norm.  A  vector  of  ones  In 


any  real  Euclidean  space  will  be  denoted  by  e.  For  a  differentiable  func¬ 
tion  g:  Rn  -*■  Rro,  Vg(x)  will  denote  the  mxn  Jacobian  matrix  evaluated 

at  the  point  x  In  Rn.  For  a  subset  I  c  {1, . m>,  gj(x)  or  gicI(x) 

will  denote  those  components  g^(x)  such  that  lei.  Similarly  Vgj(x) 
will  denote  the  rows  (Vg(x))^  of  Vg(x)  such  that  1  e  I.  The  set  of 
vectors  In  Rn  with  nonnegative  components  will  be  denoted  by  r”  . 
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2.  Exact  Penalty  Characterization  Assuming  Solvability  of  the  Minimization 


Problem 

In  this  section  we  completely  characterize  solutions  of  the  minimiza¬ 
tion  problem  (1.1)  In  terms  of  minimi zers  of  the  exact  penalty  function 
(1.2)  for  a  single  value  of  the  penalty  parameter  exceeding  the  threshold 
c^.  This  Is  done  under  the  assumptions  that  the  minimization  problem  Is 
solvable  and  that  It  satisfies  the  relaxed  Slater  constraint  qualification 
(1.6).  The  necessity  part  of  the  following  result  Theorem  2.1  Is  an  Improve 
ment  over  both  [6,  Theorem  4.9]  and  Zangwlll's  Theorem  [11,  p.  356]  both  of 
which  require  the  Slater  constraint  qualification  (1.4)  Instead  of  the 
relaxed  qualification  (1.6)  needed  here.  This  Is  a  simple  but  Important 
difference  because  It  allows  us  to  handle  linearly  constrained  problems  with 
no  constraint  qualification,  and  because  Zangwlll's  threshold  value  a, 
becomes  Infinite  under  the  relaxed  constraint  qualification  (1.6).  The  new 
sufficiency  part  of  Theorem  2.1  again  Improves  over  Zangwlll's  sufficiency 
result  by  using  the  relaxed  Slater  constraint  qualification  (1.6)  Instead  of 
the  Slater  constraint  qualification  (1.4),  and  the  smaller  threshold  value 
02  Instead  of  Oj.  It  is  Interesting  to  note  that  the  sufficiency  part  of 
Theorem  2.1  for  the  threshold  value  does  not  appear  to  have  been  given 
before  even  under  the  Slater  constraint  qualification.  Now  we  state  our 
result. 

2.1  Theorem  (Exact  penalty  characterization  of  solvable  convex  programs) 

Let  f:Rn  -►  R  and  g:Rn  Rm  be  convex  functions  on  Rn.  Let  either 
(x,u) c  Rn x R™  be  a  Karush-Kuhn-Tucker  saddlepoint  of  the  minimization 
problem  (1.1),  or  let  the  relaxed  Slater  constraint  qualification  (1.6) 


hold  and  x  be  a  solution  of  (1.1).  A  necessary  (sufficient)  condition 
for  X(Rn  to  solve  the  minimization  problem  (1.1)  Is  that  x  minimizes 
P(x,o)  over  x  In  Rn  for  each  (some)  a>  IjQIL  (o>  Hull*)  where 
OcRl  Is  any  (some)  dual  optimal  multiplier  for  (1.1). 

Proof  (Necessity)  By  assumption  or  by  [10,  Theorem  28.2]  there  exists  a 
u«R^  such  that  (x,u)  Is  a  Karush-Kuhn-Tucker  saddlepolnt  of  (1.1). 

For  any  other  dual  optimal  multiplier  D,  (x,G)  Is  also  a  Karush-Kuhn- 
Tucker  saddlepolnt  of  (1.1)  [4,  p.  5].  Hence  for  x  «  Rn  and  a  £  Hull*, 

P(x,a)  ■  f(x)  ■  f(x)  +  ug(x)  <  f(x)  +  ug(x) 

<  f(x)  +  ug(x)+  <  f(x)  +  Hull.  |ig(x)+||1  <  P(x,a) 

(Sufficiency)  Let  u t  R™  be  some  dual  optimal  multiplier  for  (1.1). 
Since  (x.G)  Is  a  Karush-Kuhn-Tucker  saddlepolnt  for  (1.1)  It  follows  by 
the  necessity  part  of  this  theorem  that  for  3:«  || u !!«, 

P(x,B)  »  min  P(x,8) 
x«Rn 


Let  x  be  a  solution  of  min  P(x,o)  for  some  a  >  Hull,,,  *  8. 


x«R 


Hence 


and 


f (x)  +  aeg(x)+  >  f(x)  +  aeg(x)+ 
f(x)  +  6eg(x)+  >  f(x)  +  8eg(x)+ 


Addition  of  the  last  two  inequalities  gives  upon  noting  that  g(x)+  ■  0 


(a-  B)eg(x)  <  0 


Since  a  >  $  this  Implies  that  g(x)  <_  0  and  hence  x  Is  feasible  for 
(1.1).  For  any  other  feasible  point  x 


f (x)  »  P(x,o)  >  P(x,a)  »  f(x).  □ 

The  following  corollary  shows  that  under  the  Slater  constraint  qual¬ 

ification  the  threshold  value  o^:*  IJuH^  of  Theorem  2.1  is  smaller  than 
that  of  Zangwlll's  Oj  as  defined  In  (1.3). 

2.2  Corollary  Let  f:Rn  -*•  R  and  g:Rn  Rm  be  convex  functions  on  Rn, 

let  x1  be  any  point  In  Rn  satisfying  the  Slater  constraint  qualification 
g(x^ )  <  0,  and  let  x  be  a  solution  of  the  minimization  problem  (1.1). 

Then  for  any  dual  optimal  multiplier  u  «  R™  for  (1.1) 

(z-i)  nail.  <  iiuiK  <  ~  ft»>  <  fl?1)  -  f(»>  * 1  a, 

min  -  g,(x‘)  min  -  g. (x1) 
l<1<m  1  l<i<m  1 

Proof  Since  x  is  a  solution  of  (1.1)  and  the  Slater  constraint  qualifica¬ 
tion  Is  satisfied  it  follows  that  x  and  some  ueR®  constitute  a  Karush- 
Kuhn-Tucker  saddlepoint  for  (1.1)  and  by  [4,  p.  5]  so  does  (x,u). 
Consequently 

f(x)  -  f(x)  +  Gg(x)  <  f(x] )  +  ug(x1 )  <  f(x] )  -  ||u||.  min  -g^x1) 

l<i<m 

from  which  (2.1)  follows.  0 

We  establish  now  another  upper  bound  for  the  threshold  value  0^:=  || u H^. 
of  Theorem  2.1  under  the  relaxed  Slater  constraint  qualification  (1.6). 


2.3  Corollary  Let  f:R  -*•  R  and  g:R  -*■  R  be  differentiable  convex 
functions  on  Rn,  let  x^  be  any  point  In  Rn  satisfying  the  relaxed 
Slater  constraint  qualification  (1.6),  and  let  x  be  a  solution  of  the 


minimization  problem  (1.1).  Then  there  exists  a  dual  optimal  multiplier 


u  c  R™  for  (1 .1 )  such  that 


(2.2) 


||®IL<  II u || i  < 

*"  1  min  -g .(x2) 

1eIl 


Id, 


max 

g(x)<0 

I(x)£J(x) 


VT(x)(A1(x)AI(x))  I, 


where 


(2.3) 


J(x)  *  {I  lcl2,  AjX  *  bj,  A1eI  1  in .  indep.} 


and  gT  (x)  *  A.  x  -  bT  , 
l2  2  2 


Proof  Since  x  Is  a  solution  of  (1.1)  and  the  relaxed  Slater  constraint 

m 


qualification  Is  satisfied  it  follows  that  x  and  some  u e  R+  constitute  a 
Karush-Kuhn-Tucker  saddlepoint  for  (1.1).  Since  f  and  g  are  differen¬ 
tiable  it  follws  that 


(2.4)  Vf(x)+uT  VgT  (x)  +  uT  At  -0,  ug(x)  =  0,  g(x)<0,  u>0 

M  X1  l2  l2 

By  the  fundamental  theorem  on  the  existence  of  basic  feasible  solutions 
[3,  Theorem  2.11]  it  follows  that  there  exists  u  e  r’J  such  that  (x,u)  is 
a  Karush-Kuhn-Tucker  saddlepoint  of  (1.1)  and 
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(2.5)  Vf(x)  +  u^Vg^Cx)  +  QI(X)AI(X)  =  0,  u.^uI(5)  *  0 
where  I (x)  belongs  to  J(x)  as  defined  by  (2.3).  Hence 

(2.6)  “l(x)  '  -<7f<5>  *  3r173l1<i»AI(x)<AI(x)AI(x))'1 

Consequently 

(2.7)  l|5I(;)||,  <  (|lVf(x)||1  tllu,  II^IVJ,  (x)||1)||*J(.)(*I(S)Aj(;)r1||1 

From  the  saddlepoint  property  we  have  that 

f(x)<f(x2)  +  Uj  gj  (x2)+GI(5)gI(-)(x2)<f(x2)-||uI  ||,  min-g^x2) 

1  1  €  I-j 

Hence 

(2.8)  ||8.  ||  < 

*1  1  min  -  g4 (x2) 

LI, 

Combining  (2.7)  and  (2.8)  gives 


Inequality  (2.2)  follows  from  the  above  upon  replacing  the  last  term  by  its 
maximum  over  all  feasible  x.  □ 


It  is  evident  that  the  last  term  in  (2.2)  may  be  difficult  to  compute 
because  of  its  combinatorial  aspect.  However  if  there  are  only  a  few  linear 
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2 

constraints,  or  If  the  point  x  Is  Interior  to  most  of  the  linear  con¬ 
straints,  in  which  case  these  constraints  can  be  lumped  with  the  nonlinear 
constraints.  It  may  not  be  too  difficult  to  compute  the  bound  of  (2.2). 
Obviously  since  x  Is  unknown  beforehand,  f(x)  must  be  replaced  by  a 
lower  bound  (as  must  be  done  for  Zangwlll's  bound  Oj )  and  ||Vf(x)||^  and 
|| Vgj  (x)  || i  •  by  upper  bounds  In  (2.2). 


3.  Exact  Penalty  Characterization  Without  Assuming  Solvability  of  the 
Minimization  Problem 

In  this  section  we  characterize  solutions  of  the  minimization  problem 

(1.1)  in  terms  of  minimizers  of  the  exact  penalty  function  (1.2)  without  any 
a  priori  assumption  regarding  the  existence  of  solutions  to  (1.1)  as  was  the 
case  in  the  previous  section.  We  do  however  need  the  relaxed  Slater  con¬ 
straint  qualification  (1.6)  and  a  mild  asymptotic  constraint  qualification 

(3.2)  below,  which  is  automatically  satisfied  If  all  the  constraints  are 
linear.  It  is  interesting  to  note  that  the  threshold  value  of  the 
penalty  parameter  In  Theorem  3.1  below  exceeds  or  equals  the  threshold  value 
^2:*  || 5 He  of  Theorem  2.1. 

3.1  Theorem  (Exact  penalty  characterization  of  feasible  convex  programs) 

Let  f:Rn  -*•  R  and  g:Rn  Rm  be  differentiable  convex  functions  on  Rn. 

Let  the  relaxed  Slater  constraint  qualification  (1.6)  hold,  let 


(3.1) 


0  t  B:*  sup  { |1  Vf  (x) ||  1  |g(x)  <  0}  < 


and  let  the  following  asymptotic  constraint  qualification  [8]  hold: 

/For  each  nonempty  I={l,....,m}  and  each  sequence  of  points 

{x1}  such  that:  g(x7)^0,  gj(x*)  =  0  and  Vg j (x^)  are  linearly 

independent,  each  accumulation  point  (VgT  ,  Vg T  ,VgT  )  of  the 

*0  A1  a2 

sequence  (Vg^^x1  )/ll ) l|,  Vg^x1),  Vg^x1 )}  satisfies 


(3.2)1 


VgT  z>0,  VgT  z>0,  VgT  z>0  for  some  z  c  Rn 
A0  *1  l2 

where  Iq u  1^  u  Ig  is  a  partition  of  I  such  that  the  sequence 
{Vgj(x7)}  is  unbounded  for  j  «  Iq  and  bounded  for  j  e  1^ , 
\9jd0  u  I1  1s  nonlinear  and  g^j  is  linear. 


A  necessary  (sufficient)  condition  for  x«R  to  solve  the  minimization 
problem  (1.1)  Is  that  x  minimizes  P(x,ct)  over  x  In  Rn  for  all 
ct  _>  ci3  (some  a  >  a3)  where 


(3.3)  o3:»  6  sup^  (llwjH^lgip)  <  0,  Wj  >  0,  gj(p)  »  0,  HwjVgjfp)^  »  1 , 

^gjel(P)  Ind..  1^(1#.... .m) ) 


Proof  Ue  first  note  that  the  finiteness  of  ^  Is  ensured  by  the  asymp¬ 
totic  constraint  qualification  [8,  Theorem  2.1]. 

(Necessity)  Let  x  be  a  solution  of  (1.1)  and  let  iltl®  be  an 
optimal  dual  multiplier  for  (1.1).  We  will  show  that  a3  £  || u and 

hence  by  the  necessity  part  of  Theorem  2.1,  x  minimizes  P(x,a)  for 

a  >,  a3.  If  Vf(x)  *  0  we  take  u  •  0  and  evidently  Oj  >  ||u ||M  *  0. 

Suppose  now  Vf(x)  t  0.  Take  u  ■  (uL,  uK)  where  uL  >  0  and  corre¬ 

sponding  to  "basic"  9jeL(*)  *  0  such  that  Vgj«L^  are  11nearly  1"- 
dependent  and  uK  *  0.  Hence  by  the  Karush-Kuhn-Tucker  conditions  [7] 

Vf (x)  +  uLVgL(x)  -  0 

and  consequently 

1 1V?TXTIT7  v*l£5)||i  * 1 


Hence  by  the  definition  (3.3)  of  a3  and  the  definition  (3.1)  of  6 


IHL 

87TWTin- 


II5IL 


(Sufficiency)  Let  x  be  a  solution  of  min  P(x,a)  for  some  a  >  a,. 

x«Rn 

We  first  show,  by  contradiction,  that  g(x)  <  0.  For  If  x  is  Infeasible 
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then  by  [8,  Theorem  2.1]  there  exist  a  feasible  p(x)  such  that 

(3.4)  ll*-P(X)IL<xll9(*>Jl  “  T  *9(5)+ 

Then  for  a  > 

f(p(x))  -  P(p(x).ct) 

>  P(x, a)  (Since  x  minimizes  P(x,a)  over  xeRn) 

-  f(x)  +  aeg(x)+ 

>  f (x)  +  8||x-p(x)|L  (By  (3.4),  a>a3  and  g(x)+*0) 

>  f(x)  +  ||Vf(p(x))||i  ||x-p(x)IL  (By  (3.1)) 

>,  f(x)  -  vf(p(x))  (x-p(x))  (By  the  Cauchy- Schwarz  Inequality) 

>  f(p(x))  (By  the  convexity  of  f) 

which  Is  a  contradiction.  Hence  g(x)  <  0  and  x  Is  feasible.  For  any 
other  feasible  x  and  a  >  a3 

f (x )  ■  P(x,a)  <_  P(x,a)  *  f(x) 

and  hence  x  solves  (1.1).  □ 

Obviously  the  threshold  value  o3  given  by  (3.2)  Is  difficult  to 
compute  In  general.  However  besides  providing  an  existence  result  for  the 
minimization  problem  (1.1),  It  Is  useful  to  know  that  such  a  threshold 
value  exists  and  to  know  how  It  depends  on  the  problem  parameters,  espe¬ 
cially  when  one  is  engaged  In  an  unconstrained  exact  penalty  function 
minimization  either  on  Rn,  as  a  substitute  for  the  original  constrained 
optimization  problem,  or  on  R^  as  part  of  an  iterative  method  [5]. 
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4.  An  Application:  The  Blg-M  Method  for  Convex  Programs 

In  linear  programming,  a  well  known  method  [9,  1]  for  solving  a 
linear  program  without  an  explicit  phase  I  procedure  Is  to  add  nonnegative 
artificial  variables  to  the  constraints  and  then  add  a  penalty  to  the 
objective  function  Involving  the  artificial  variables.  If  the  penalty 
parameter  Is  "sufficiently  large",  then  the  artificial  variables  will  be 
driven  to  zero  and  an  optimal  solution  will  be  obtained.  If  one  exists. 

In  this  section  we  will  make  the  "sufficiently  large"  concept  precise  by 
using  the  results  of  the  two  previous  sections  and  extend  the  Idea  of  the 
b1g>M  method  to  convex  programs.  We  first  state  a  simple  lemma  whose 
elementary  proof  we  omit. 


4.1  Lemma  Let  f:Rn  -*■  R,  g:Rn  -*•  Rw  and  let  a  >  0.  Then  the  problems 


(4.1)  min  f(x)  +  aeg(x).  ■:  min  P(x,a) 

x£Rn  xcRn 

(4.2)  min  f(x)  +  aez  s.t.  g(x)  <  z,  z  >  0 

(x.zWT1 

are  equivalent  In  the  following  sense:  For  each  solution  x  of  (4.1), 

(x,  z:»  g(x)+)  solves  (4.2),  and  for  each  solution  (x,z)  of  (4.2),  x 
solves  (4.1). 

The  formulation  of  (4.2)  Is  the  blg-M  formulation  and  Is  used  In 
linear  programming  because  It  Is  easy  to  obtain  a  feasible  point  for  It  by 
taking  any  x  In  Rn  and  z:«  g(x)+.  Formulation  (4.2)  can  be  used  also 
for  the  very  same  reason  In  convex  programming.  Theorem  2.1  tells  us  that 
If  we  know  a  priori  that  problem  (1.1)  has  a  solution,  f  and  g  are 
convex  and  the  relaxed  Slater  constraint  qualification  (1.6)  Is  satisfied 


then  the  penalty  parameter  a  of  the  big-M  formulation  (4.2)  must  satisfy 
a  >  ctg :*  ||u||„  where  u  Is  any  optimal  dual  multiplier  to  (1.1).  Note 
that  If  g  Is  linear,  then  the  relaxed  Slater  constraint  qualification 
(1.6)  Is  satisfied  by  any  feasible  point  x.  If  we  have  no  a  priori  know¬ 
ledge  that  (1.1)  Is  solvable,  but  that  It  Is  merely  feasible,  that  f,  g 
are  differentiable  and  convex,  and  that  (3.1)  and  the  constraint  qualifi¬ 
cations  (1.6)  and  (3.2)  are  satisfied  then  the  penalty  parameter  a  of  the 
blg-M  method  (4.2)  must  satisfy  a  >  ci3  where  a3  Is  defined  by  (3.3). 
Note  that  If  g  Is  linear  then  (1.6)  and  (3.2)  are  automatically  satisfied 
and  If  In  addition  f  Is  nonconstant  and  linear,  then  (3.1)  is  also  automat 
Ically  satisfied. 
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